The present invention pertains to semiconductor based image sensors and more particularly to Active Pixel image sensors (APS) or Passive Pixel image sensors (PPS) having increased dynamic range.
APS are solid state imagers wherein each pixel contains both a photo-sensing means and at least one other active component, creating a charge that is converted to a signal (either a voltage or current signal). The signal represents the amount of light incident upon a pixel photosite. The dynamic range (DR) of an image sensing device is defined as the ratio of the effective maximum detectable signal level, typically referred to as the saturation signal, (Vsat), with respect to the rms. noise level of the sensor, ("sgr"noise) This is shown in Equation 1.
Dynamic Range=Vsat/"sgr"noisexe2x80x83xe2x80x83Equation 1
Image sensor devices such as charge coupled devices (CCD) that integrate charge created by incident photons have dynamic range limited by the amount of charge that can be collected and held in a given photosite, (Vsat). For example, for any given CCD, the amount of charge that can be collected and detected in a pixel is proportional to the pixel area. Thus for a commercial device used in a megapixel digital still camera (DSC), the number of electrons representing Vsat is on the order of 13,000 to 20,000 electrons. If the incident light is very bright and creates more electrons that can be held in the pixel or photodetector, these excess electrons are extracted by the anti-blooming means in the pixel and do not contribute to an increased saturation signal. Hence, the maximum detectable signal level is limited to the amount of charge that can be held in the photodetector or pixel. The DR is also limited by the sensor noise level, "sgr"noise. Due to the limitations on Vsat, much work has been done in CCD""s to decrease "sgr"noise to very low levels. Typically, commercial megapixel DSC devices have a DR of 1000:1 or less.
The same limitations on DR exist for APS devices. The Vsat is limited by the amount of charge that can be held and isolated in the photodetector. Excess charge is lost. This can become even more problematic with APS compared to CCD due to the active components within the pixel in the APS, limiting the area available for the photodetector, and due to the low voltage supply and clocks used in APS devices. In addition, since APS devices have been used to provide image sensor systems on a chip, the digital and analog circuits used on APS devices such as timing and control and analog to digital conversion, that are not present on CCD""s, provide a much higher noise floor on APS devices compared to CCD. This is due to higher temporal noise as well as possibly quantization noise from the on-chip analog to digital converter.
Within the art of semiconductor based image sensors there are numerous disclosures that provide extended dynamic range for both APS and PPS devices. These include (1) measurement of number of clock periods to reach a threshold as taught by Konuma in U.S. Pat. No. 5,650,643; (2) capture of 2 or more correlated images with varying integration times as described by Orly Yadid-Pecht et al. in xe2x80x9cWide Intrascene Dynamic Range CMOS APS Using Dual Samplingxe2x80x9d published in the 1997 IEEE Workshop on Charge-Coupled Devices and Advanced Image Sensors; paper #R15; (3) logarithmic transfer function pixel architectures as described by Sypros Kavadias et al. in xe2x80x9cOn-chip Offset Calibrated Logarithmic Response Image Sensorxe2x80x9d, 1999 IEEE Workshop on Charge-Coupled Devices, and Advanced Image Sensors, pp. 68-71, M. Loose et al., xe2x80x9cSelf-Calibrating Logarithmic CMOS Image Sensor with Single Chip Camera Functionalityxe2x80x9d, 1999 IEEE Workshop on Charge-Coupled Devices and Advanced Image Sensors, pp. 191-194, N. Ricquier, et al., xe2x80x9cActive Pixel CMOS Image Sensor with On-Chip Non-Uniformity Correctionxe2x80x9d, 1995 IEEE Workshop on Charge-Coupled Devices and Advanced Image Sensors, . J. Huppertz et al., xe2x80x9cFast CMOS Imaging with High Dynamic Rangexe2x80x9d, 1997 IEEE Workshop on Charge-Coupled Devices and Advanced Image Sensors, pp. 1-4; and (4) varying the level of the reset gate during integration as described in prior art references by S. Decker et al., xe2x80x9cComparison of CCD and CMOS Pixels for a Wide Dynamic Range Area Imagerxe2x80x9d, 1995 IEEE Workshop on Charge-Coupled Devices and Advanced Image Sensors, and xe2x80x9cA 256xc3x97256 CMOS Imaging Array with Wide Dynamic Range Pixels and Column-Parallel Digital Outputxe2x80x9d, IEEE Journal of Solid State Circuits, Vol 33, No. 12, December 1998.
U.S. Pat. No. 5,650,643 issued to Konuma (Konuma) teaches a device that can be used to increase the dynamic range of solid state image sensing devices. Konuma shows a means to increase the effective Vsat level by incorporating a comparator and a counter associated with a photodetector to measure the time required to reach an integrated signal threshold level, and provide this as the only sensor output. The counter is used in conjunction with the comparator to determine the number of counter clock periods that it took the photodetector to reach the signal level of that supplied to the comparator input. The device then provides only the number of counter clock periods as an output or signal value associated with the photodetector.
While the disclosure of Konuma does provide increased dynamic range through effectively increasing Vsat, this approach has several problems.
First, if one were to have a counter and comparator in each pixel, the number of components in each pixel is very large leading to a pixel with a very small fill factor or a very large pixel. This approach is not practical given the present minimum feature sizes of state of the art semiconductor technology, and the need for small pixel, low cost image sensors.
Second, the output for each pixel is a counter value for the time required to reach a given threshold, and does not contain an analog output value for the actual amount of charge integrated in the photodetector. With this approach, although the effective Vsat level is increased, the effective DR will be limited by the time period or accuracy of the counter clock, and the size of the counter. For example, if the counter has 10 bits or 1024 counts, the dynamic range is extended to 10 bits provided that the 1024 master clock periods can fit into the desired exposure time. If the desired exposure time were 100 msec., then the counter clock period must be xe2x89xa697.6 usec. If one were try to extend the DR to 20 bits, a 20 bit counter would be required, and a counter clock frequency of  greater than 10.5 MHz for an exposure time of 100 msec. In this example, the extension of the DR from 10 to bits results in a clock frequency requirement that is 1000 times faster. As the exposure time decreases a commensurately faster master clock is required. For example, if an exposure time of {fraction (1/60)}th of a second were desired or required in the case of capturing an image in outdoors in bright sunlight, a master clock of 63 MHz would be required to quantize 20 bits. It is evident that very fast counter clocks are required to provide high dynamic range in typical exposure conditions. Also, as the number of bits in the counter gets larger, more area required to integrate this into the pixel, producing a larger and larger pixel. Typical counters require 4-8 transistors per bit. Thus a 20 bit counter would require 80-160 transistors, yielding pixel sizes of  greater than 40 um in a 0.35 um CMOS process. Additionally this approach requires that all pixels within the image sensor reach the programmed threshold level in order to have an output value for each pixel. This would require very long exposure times to allow dark regions of the scene to reach the threshold level if the threshold level is near Vsat. The exposure times could be decreased by programming the threshold level to a very low value, but this would reduce the accuracy of information in very bright regions of the scene since they will reach the threshold value in extremely short time periods.
Thirdly, with the approach of Konuma, at the brightest light levels the data is more quantized. This is shown is Equation 2 by looking at how the effective light measurement is calculated from the time to threshold.
If one knows the amount of time (tT) required to reach a threshold (VT) and assume that the source is constant over the time being measured, then one can calculate the amount of light at any arbitrary time, (tM). The expression for the extended effective voltage (Vext) is given by Equation 2 below.                               V          ext                =                                                            V                T                            ·                              t                M                                                    t              T                                ❘                                    Equation        ⁢                  xe2x80x83                ⁢        2            
In a discrete system the time variable, tT, would be measured by a quantized unit as indicated in Equation 3.                               t          T                =                                                            t                M                            ·              cv                        MaxCv                    ❘                                    Equation        ⁢                  xe2x80x83                ⁢        3            
Where cv is the quantized integer code value and MaxCv is the code value that corresponds to the cv value at tM. Substituting values we arrive at Equation 4.                               V          ext                ⁢                  xe2x80x83                =                  xe2x80x83                ⁢                                                            V                T                            ·              MaxCv                        cv                    ❘                                    Equation        ⁢                  xe2x80x83                ⁢        4            
Referring to FIG. 2, a code value (cv) of zero implies infinite light. The first measurable quantization, which is also the largest, is between cv=1 and cv=2. The quantization for an 8 bit linear system is 0.0039, which is less than the smallest quantization in a time to threshold method described by Konuma.
Fourthly, if one were to have a single counter and comparator used outside of the pixel array to keep track of the time to threshold, each pixel would then have to be measured at an extremely high rate in order to have a small enough sampling frequency per pixel to provide fine enough quantization for extension of the dynamic range. For example, assume that 10 bits of quantization over the desired exposure time is required, and that there are 1 million pixels in the image sensor. Given a desired exposure time of 100 msec., each pixel would have to be accessed and measured against the programmed threshold level every 97.65 usec. This means that 1 million pixels need to be sampled every 97.65 usec. This would require a pixel sampling rate of 1 pixel per 97.65 psec, or 10.24 GHz. A means for doing this is not disclosed by Konuma or elsewhere in the field of APS devices or other image sensor devices.
Finally, the output value provided is a time. In order to reconstruct the incident image from this output, (i.e. determine the signal level), one must extrapolate by multiplication from the time value. This can degrade the effective noise level of the sensor. The value t is used to measure the time for a voltage v(t) to reach to a threshold. The signal VPD(t) represents the accumulation of photons over time with some Gaussian additive noise with a standard deviation of "sgr"v. One experienced in the art can show that the noise in the extended voltage domain ("sgr"Ext) is related to the additive noise as indicated by Equation 5.                               σ          Ext                =                                            2              ·                              σ                v                            ·                              t                M                            ·                              V                T                2                                                                    t                T                            ·                              (                                                      V                    T                    2                                    -                                      σ                    v                    2                                                  )                                              ❘                                    Equation  5            
Given that tM is always greater than tT one can see that the value of "sgr"Ext is always greater than "sgr"v. From the foregoing discussion it should be apparent that there remains a need within the prior art for a device that provides extended Vsat and dynamic range while retaining low noise, small pixel, simple and low frequency readout, and means to manage the quantization of extended voltage signals.
With the method of capturing two or more frames as disclosed by prior art references: Orly Yadid-Pecht et al. in xe2x80x9cWide Intrascene Dynamic Range CMOS APS Using Dual Samplingxe2x80x9d; 1997 IEEE Workshop on Charge-Coupled Devices and Advanced Image Sensors; paper #R15; O. Pecht et al., xe2x80x9cCMOS APS with Autoscaling and Customized Wide Dynamic Rangexe2x80x9d, 1999 IEEE Workshop on Charge-Coupled Devices and Advanced Image Sensors; and M. Schantz et al., xe2x80x9c256xc3x97256 CMOS Imager with Linear Readout and 120 dB Dynamic Rangexe2x80x9d, 1999 IEEE Workshop on Charge-Coupled Devices and Advanced Image Sensors, two or more frames of the same image or scene with different integration times are captured, multiple readouts are required and the integration times for each readout must be chosen in accordance with the scene illuminance in order to get an accurate interpolation. This adds complexity to the imaging auto-exposure system to facilitate rapid choice of integration times and has not been shown in the art. Also, additional frame memory is required to perform the multiple frame output comparison and calculation of effective signal level. Additionally, if there is any motion or change in scene illuminance between the 2 frames, this method of extending dynamic range will not work.
With the methods of extending dynamic range described by: Sypros Kavadias et al., xe2x80x9cOn-chip Offset Calibrated Logarithmic Response Image Sensorxe2x80x9d, 1999 IEEE Workshop on Charge-Coupled Devices and Advanced Image Sensors, pp. 68-71; M. Loose et al., xe2x80x9cSelf-Calibrating Logarithmic CMOS Image Sensor with Single Chip Camera Functionalityxe2x80x9d, 1999 IEEE Workshop on Charge-Coupled Devices and Advanced Image Sensors, pp. 191-194; and N. Ricquier, et al., xe2x80x9cActive Pixel CMOS Image Sensor with On-Chip Non-Uniformity Correctionxe2x80x9d, 1995 IEEE Workshop on Charge-Coupled Devices and Advanced Image Sensors, a pixel with a logarithmic transfer function is provided. This approach of using a MOSFET sub-threshold exhibits behavior having very high fixed pattern noise. Approaches to correct this require extra system memory, individual threshold trimming of each pixel, or extra transistors per pixel. This increases chip size as well as system cost and complexity.
With the methods of extending dynamic described by J. Huppertz et al., xe2x80x9cFast CMOS Imaging with High Dynamic Rangexe2x80x9d, 1997 IEEE Workshop on Charge-Coupled Devices and Advanced Image Sensors, pp. 1-4; and S. Decker et al., xe2x80x9cComparison of CCD and CMOS Pixels for a Wide Dynamic Range Area Imagerxe2x80x9d, 1995 IEEE Workshop on Charge-Coupled Devices and Advanced Image Sensors, the reset gate voltage level is changed from an on-state to an off-state with a specified time response during integration. With this method the dynamic range is extended by skimming off charge to the reset drain for bright pixels so that the pixel does not saturate. This method has the disadvantages of requiring complicated timing to operate in an electronic shutter mode, and difficulty in discerning whether or not a particular signal level is due to skimming of charge or is simply the total integrated signal level.
From the foregoing discussion it should be apparent that there remains a need within the prior art for a device that retains provides extended dynamic range while retaining low noise, small pixel, single frame readout, and flexible transfer function.
According to the present invention, there is provided a solution to problems of the prior art. In the present invention, blooming behavior of CMOS based image sensors is used to provide extended intrascene dynamic range.
A first embodiment of the present invention utilizes the charge that blooms from the photodetector during integration. In prior art devices the signal level of the photodetector is limited in bright regions by removing and discarding the blooming charge. In the present invention the charge that blooms from the photodetector is integrated for a period that is controlled independently from the photodetector integration time, and this collected blooming charge is added to the photodetector signal charge in the readout of the pixel.
A second embodiment of the present invention provides extended dynamic range by utilizing the behavior of diminishing signal with increasing light level associated with the readout of CMOS APS under high illumination levels. This is done by performing 2 samples of the reset level that have different elapsed times between reset and measurement of the reset level, thus providing 2 different effective reset level integration times. By comparing the 2 measured reset levels, one can determine the effective signal level of the photodetector beyond the physical saturation signal of the photodetector. The difference determined in the two reset measured levels is used to if the pixel is operating in the diminishing high light signal region, or in low light standard linear signal region. Depending on the result, a different transfer function is applied to voltage output from the from readout of the photodetector of that pixel to provide an effective signal level for that pixel. If the light that is incident on that pixel is bright, and a difference in the 2 measured reset levels exceeds a predetermined threshold, a transfer function associated with the diminishing signal region will be used to calculate the photodetector value. If a difference in the 2 measured reset levels does not exceed a predetermined threshold value, then the standard linear transfer function will be used to identify that photodetector value. Additionally, the value of the measured difference can be used independently to calculate or determine the effective signal level or incident illumination level for that pixel.
According to the present invention, an active pixel sensor device that significantly increases the dynamic range of the device and can be used in current system designs is provided by an X-Y addressable MOS imager sensor method and apparatus wherein a semiconductor based MOS sensor having an array of pixels forming the X-Y addressable MOS imager, the X-Y addressable MOS imager having a plurality of the pixels such that each pixel within the plurality of pixels has a photodetector with a reset mechanism that adjusts the photodetector potential to a predetermined potential level employs the measuring a plurality of reset levels with two different elapsed times between reset and measurement of the reset level, a comparison circuit operatively coupled to the means for measuring to determine a difference in reset levels, a predetermined set of transfer functions used to identify effective signal levels of the photodetectors, and determines from the difference which transfer function is applicable to that photodetector range of accumulated light. In response to the difference detected, transfer functions are applied to the charge read out from the photodetector. The transfer functions comprise a first transfer function that is pre-photodetector saturation function and a second transfer function that is a post-photodetector saturation transfer function. These transfer functions are applied to two reset levels of the photodetector within the same row readout period. The response to the difference can be the application of a series of digital adds and digital multiplies. The difference between the 2 reset levels is used to calculate the effective pixel signal level from the difference.
The invention has the following advantages. It provides for extending the dynamic range of a sensor that can easily be employed within current pixel and sensor architectures with little or no modification. Small pixel with high fill factor can detect signal level out to 40,000X Vsat single frame capture standard read-out.